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ABSTRACT 


This  paper  is  to  be  used  as  a  guide  in  gathering  and  treating  data 
to  be  used  in  comparing  two  chemical  test  methods  to  determine 
equivalency.  The  concept  of  an  equivalent  test  method  is  well 
defined  in  terms  of  repeatability  and  reproducibility  as  well  as 
accuracy  and  precision.  All  statistical  tests  and  calculations 
are  explained  and  illustrated  using  numerical  examples. 


It  should  be  noted  that  this  is  a  preliminary  report  subject  to 
modification  after  finalized  reviews  of  interested  parties. 


INTRODUCTION 


DETERMINATION  OP  THE  EQUIVALENCY  OF  TWO 
CHEMICAL  TEST  METHODS 

■/ 

This  paper  is  designed  as  a  statistical  guide  for  determining  the 
equivalency  of  two  chemical  test  methods.  A  variety  of  tests  are 
performed  on  the  data  which  is  obtained  by  two  methods: 

a.  Method  1  is  the  specified  method  that  has  been  referred  to 
in  the  specification. 

b.  Method  2  is  the  proposed  method  which  is  being  compared  to 
the  specified  method  to  determine  if  it  is  equivalent  or  not. 

The  term  equivalency  must  be  defined  for  this  paper.  In  general 
the  equivalency  of  two  test  methods  implies  that  the  methods  have 
the  same  accuracy  and  the  same  precision.  This  is  the  only  case 
in  which  the  two  test  methods  may  be  deemed  equivalent.  If  method 
1  is  more  or  less  accurate  or  precise  than  method  2,  then  the 
methods  are  not  equivalent. 

This  guide  is  divided  into  four  parts.  The  first  two  parts 
represent  data  which  must  be  gathered  and  treated  by  the  proponent 
of  the  new  test  method.  The  results  obtained  in  this  section  will 
provide  guidance  as  to  whether  a  round  robin  should  even  be 
performed.  If  the  results  obtained  in  the  first  two  parts 
indicate  that  the  proposed  test  method  may  be  an  equivalent 
method,  then  the  round  robin  should  be  organized  and  the  data 
obtained  for  treatment  in  Parts  3  and  4. . ^ 

The  data  used  in  Parts  3  and  4  is  essentially  data  from  five  labs 
done  on  three  samples.  This  data  is  used  for  evaluating  the 
repeatability  and  reproducibility  associated  with  each  method. 

In  statistical  terms  the  two  methods  may  be  considered  equivalent 
if  they  have  equal  means  and  variances.  That  is,  the  methods  are 
equivalent  if  we  can  demonstrate  that  the  sample  means  and 
variances  generated  by  each  method  are  from  the  same  population. 
Another  criteria  is  that  the  lab  to  lab  variation  be  the  same  for 
both  methods. 

In  conjunction  with  the  determination  of  the  equivalency  of  both 
methods,  this  paper  illustrates  the  determination  of  each  methods 
repeatability  and  reproducibility  using  analysis  of  variance 
(ANOVA) .  The  conditions  necessary  to  perform  ANOVA  are  stated  and 
the  calculations  necessary  to  perform  a  one-way  ANOVA  and  a  two 
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way  ANOVA  are  illustrated.  After  these  ANOVAs  have  been 
determined,  the  method  of  using  the  components  of  variance  to 
estimate  the  method  repea tabiliity  and  reproducibility  is  shown.  . 
It  should  be  noted  that  this  paper  uses  the  method  of  ANOVA  to 
characterize  the  variability  of  each  method.  It  does  not  attempt 
to  differentiate  between  one  lab  and  the  next. 

It  should  be  emphasized  that  this  paper  is  basically  an 
instructional  guide  to  indicate  the  various  tests  and  operations 
that  can  be  performed  on  the  data  to  determine  equivalency.  The 
data  used  in  this  paper  is  ideal  since  it  has  been  fashioned  as  an 
illustrative  example.  In  each  case  the  actual  experimental  data 
obtained  using  each  method  must  be  examined  to  see  if  all 
necessary  conditions  are  fulfilled  in  order  that  the  approaches 
used  in  this  paper  can  be  utilized. 

Computer  programs  have  been  generated  which  facilitate  all  of  the 
time  consuming  calculations  which  are  required  for  ANOVA.  Data 
generated  from  these  round  robins  should  be  submitted  to 
DRDAR-QAR-R.  Sample  data  sheets,  letters  for  round  robin 
participants,  and  other  guidelines  for  the  round  robin  in  general 
are  included  in  Appendix  B. 
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PART  I 


COMPARISON  OF  TWO  ANALYTICAL  TEST  METHODS  FOR  THE 
DETERMINATION  OF  THE  AMOUNT  OP  RDX  IN  COMPOSITION  B. 


This  section  evaluates  the  accuracy  of  each  method  by  comparing 
the  results  obtained  using  a  given  analytical  method  with  the 
known  value  of  a  synthetic  sample.  It  sets  95  percent  confidence 
limits  on  the  means  generated  by  each  method  using  a  student's  "t" 
distribution.  If  the  test  is  of  the  desired  accuracy,  the  actual 
sample  value  should  fall  within  the  95  percent  confidence  interval 
generated  by  each  method.  The  average  bias  for  each  method  with 
respect  to  the  synthetic  samples  is  also  calculated. 


Preparation  of  Synthetic  Samples 


The  technician  should  weigh  accurately  to  within  +  .OOlg  fourteen 
different  synthetic  samples  of  Composition  B  adhering  to  the 
specified  composition  (not  including  the  tolerances).  A  lO.OOOg 
sample  is  used  in  the  following  examples.  Thus  the  technician 
will  have  fourteen  lO.OOOg  samples  having  the  following 
compos i ti on; 


Amount  of  RDX 
Amount  of  TNT 
Amount  of  WAX 


5.950  +  .OOlg 

3.950  +  .OOlg  synthetic  sample  1 

.100  +  .OOlg 


In  addition  to  these  samples,  eight  other  samples  should  be  made 
with  four  having  the  amount  of  RDX  at  a  low  concentration  of  RDX: 


5.000  +  .OOlg  RDX 

4.900  +  .OOlg  TNT  synthetic  sample  2 

.100  +  .OOlg  WAX 

and  four  samples  are  prepared  at  a  high  concentration  of  RDX; 


7.000  +  .OOlg  RDX 

2.900  +  .OOlg  TNT  synthetic  sample  3 

.100  +  .OOlg  WAX 

It  should  be  emphasized  that  all  synthetic  samples  should  be 
weighed  out  individually.  Thus  in  preparing  synthetic  sample  1, 
fourteen  weighings  of  each  of  the  three  components  (42  separate 
weighings)  are  required. 

In  analyzing  synthetic  sample  number  1,  seven  determinations  using 
each  method  are  required.  In  analyzing  synthetic  samples  2  and  3, 
duplicate  determinations  using  each  method  are  required.  The 
results  are  shown  in  Tables  1  and  2. 
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TABLE  1 


Data  Reported  for  RDX  Content 
in  14  Synthetic  Samples  by  2  Methods 


Synthetic  Sample  1; 

Method  1  (specified  method) 
5.960 
5.975 
5.991 
5.966 
5.936 
5.988 
5.892 
X  =  5.960 
S  =  .0363 


Method  2  (proposed  method) 
5.979 
6.068 
5.916 
6.224 
6.089 
6.193 
5.994 
X  =  6.066 
S  =  .1131 


NOTE;  All  data  has  been  inspected  for  outliers  and  none  are 
present . 


TABLE  2 

Data  Reported  for  RDX  Content 
in  8  Synthetic  Samples  By  2  Methods 


Synthetic  Sample  2; 

Method  1 
5.037 
5.006 
X  =  5.022 
S  =  .0218 


Method  2 
5.158 
5.058 
X  =  5.108 
S  =  .0707 
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Method  1 


Method  2 


6.991 

7.058 

7.043 

7.152 

X 

=  7.017 

X  =  7.105 

s 

=  .0368 

S  =  .0665 

95%  Confidence  Limits  for  the  Mean 


A  two  sided  "t"  test  is  used  to  set  confidence  limits  on  the  means 
generated  by  each  method.  From  Table  A-2  for  6  degrees  of 
freedom,  t_g75  =  2.447.  The  95%  confidence  limits  on  pare  given 
by; 


(X  -  ts/n*5)  n  ^  (X  +  ts/n*^)  where  t  =  t.975 


Method  1 


Method  2 


ts/n*5  =  2.447  (.0363)  =  .0336 
(7)  -5 

X  =  5.960 

5.926  <  p  <  5.994 

ts/n-5  =  2.447  (.1131) 

(7)  -5 

X  =  6.066 

5.961  <  II  <  6.170 
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Conclusions 

The  actual  value  ot  the  sample  composition  (  =  5.950)  is 

included  in  the  95%  confidence  interval  generated  by  Method  1  and 
IS  not  included  in  the  95%  confidence  interval  generated  by  Method 

2. 


Determination  of  the  Average  Bias 
The  bias  is  the  difference  between  the  measured  and  actual  value: 
Bias  =  Measured  Value  -  Actual  value 


For  each  method  a  bias  is  calculated  tor  each  synthetic  sample. 
The  weighted  average  bias  is  then  calculated. 

Me  mod  1 

Syntnetic  Sample  1  Bias  =  X  -  n  =  7  replicates 

=  5.9fa0  -  5.950 
=  .  010 

Synthetic  Sample  2  Bias  =  5.022  -  a. 00  n  =  2  replicates 

=  .022 


Syttthetic  Sample  3 


Bias  =  7.017  -  7.00  n  =  2  replicates 
=  .017 


Weighted  average  oias  =  7  (.010)  +  2  (.022)  +  2  (.017)  =  .0135 


Method  2 


Synthetic  Sample  i 


Synthetic  Sample  ^ 


Synthetic  Sample  3 


Bias  =  b.UOb  -  5.950  n  =  7  replicates 
=  .116 


Bias  =  5.100 
=  .108 

bias  =  7.105 
=  .105 


-  b.OOO  n  =  2  replicates 


-  7.000  n  =  2  replicates 


Weighted  average  bias  =  7  (.lib)  +  2  (.108)  +  2  (.105)  =  .113 


Conclusions 


PART  2 


TREATMENT  OF  DATA 


This  section  uses  Cochran's  Test  to  determine  the  homogeneity  of 
the  variances  obtained  among  lots  for  a  given  method.  In  order  to 
determine  the  presence  of  outliers  Dixon's  Range  Test  is  used. 
Finally  the  means  and  variances  of  each  method  are  compared  using 
"F"  tests  and  "t"  tests. 


Dixon's  Range  Test 


The  observed  values  are  first  arranged  in  increasing  or  decreasing 
order  (see  Table  3)  and  labled  to  Xn  (see  Table  4) .  The 
ratio  rj^]^  is  then  calculated  and  compared  with  the  critical 
values  in  Table  A-1  at  the  .05  signficance  level.  If  the 
calculated  value  of  r^]^  exceeds  the  value  obtained  from  Table 
A-1,  the  point  is  considered  an  extreme  observation  and  eliminated 
from  the  set  of  data. 
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IX  ITI 
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TABLE  3 

Data  Reported  for  RDX  Content  in  Three  Different  Lots 
Using  the  Proposed  Method  and  the  Specified  Method. 


Method  2 
LOT  A 

(Proposed 
LOT  B 

Method ) 

LOT  C 

Method 
LOT  A 

1  (Specified 
LOT  B 

Method) 
LOT  C 

5.742 

5.878 

6.030 

5.646 

5.449 

5.947 

5.822 

5.983 

6.070 

5.695 

5.863 

5.999 

5.860 

6.026 

6.158 

5.704 

5.899 

6.042 

5.860 

6.032 

6.165 

5.742 

5.921 

6.063 

5.865 

6.052 

6.193 

5.743 

5.938 

6.070 

5.884 

6.054 

6.199 

5.748 

5.956 

6.078 

5.899 

6.128 

6.238 

5.753 

5.959 

6.087 

5.900 

6.153 

6.271 

5.780 

5.993 

6.101 

5.932 

6.190 

6.298 

5.796 

6.008 

6.121 

5.943 

6.195 

6.304 

5.811 

6.060 

6.125 

X  = 

5.861 

X  = 

6.112 

X  = 

5.193 

X  = 
5.715 

X  = 

5.905 

X  = 
6.0633 

S  = 

.0578 

S  = 

.138 

S  = 

.0915 

S  = 
.0496 

S  = 

.169 

S  = 
.0553 
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TABLE  4 


Data  for  Lot  B  Method  1  Arranged  in  Increasing  Order 
with  Points  Labled  for  Dixons 's  Range  Test 


(increasing  order)  (decreasing  order) 


5.449 

Xl 

XlO 

5.863 

X2 

X9 

5.899 

X3 

X8 

5.921 

X4 

X7 

5.938 

X5 

H 

5.956 

X6 

X5 

5.959 

X7 

X4 

5.993 

X8 

X3 

6.008 

X9 

X2 

6.060 

xio 

Xl 

DIXON'S  RANGE  TEST  APPLIED  TO  LOT  B  METHOD  1 


For  increasing  order; 

rii  =  (X2  -  Xi)/(X9  -  Xi) 

For  decreasing  order; 
ril  =  (X2  -  Xi)/(X9  -  Xi) 


(5.863  -  5. 449)7(6.008  -  5.44)  =  .741 

(6.008  -  6.060)7(5.863  -  6.060)  =  .264 


From  Table  A-1  for  10  observations  at  .05  signficance  level,  r^ 
should  be  rejected  if  it  is  greater  than  .477.  Therefore,  point 
Xi  in  the  increasing  order  list  should  be  eliminated.  (The 
point  5.449  is  an  outlier.)  The  data  would  then  consist  of  the 
remaining  9  observations. 
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Data  (Lot  B  Method  1)  Extreme  Observation  Removed 


5.863 
5-899 
5.921 
5.938 
5.956 
5.959 
5.9^3 
6.008 
6.060 
X  =  5.955 
S  =  .0595 


All  Other  data  in  Table  3  has  been  inspected  for  outliers  and  none 
are  present. 


Cochran's  Test  for  Homogeneity  of  Variances 
Cochran's  test  uses  the  equation; 


2 

largest 


2 

where  is  the 

variance  of  Lot  A, 
B,  or  C  for  a  given 
method 


If  C  is  less  than  the  value  exceeded  5  percent  of  the  time  (i.e., 
C^95)  there  is  no  evidence  to  indicate  the  variances  are  not 
homogeneous . 
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For  Method  2: 


.1382 

C  - - -  =  .f^l9 

.0502  +  .1382  +  .09152 


For  Method  1; 


c  = 


.1692 

.04962  +  .1692  +  .05532 


.R38 


For  3  means  (k  =  3)  and  9  degrees  of  freedom  (df  =  9)  the  value  of 
C  at  the  .05  significance  level  may  be  found  in  Table  A-4. 

C.95  =  .6167  (k  =  3,  df  =  9) 


Since  both  values  of  C  calculated  for  Method  1  and  Method  2  are 
larger  than  C.95  we  reject  the  hypothesis  that  the  variances  of 
lots  A,  B  and  C  are  homogeneous.  Upon  further  inspection  of  the 
data  it  appears  that  the  variance  for  Lot  B  is  noticeably  larger 
than  the  variance  of  lots  A  and  C  for  both  methods. 

If  we  drop  the  data  for  Lot  B  and  consider  only  lots  A  and  C  we 
can  recalculate  C  for  each  method. 


For  Method  2; 

.09152 

C  =  -  =  .713 

.0502  +  .09152 


For  Method  1: 

.05532 


04962  +  .05532 


For  2  means  (k  =  2)  and  9  degrees  of  freedom  (df  =  9)  the  value  of 
C  at  the  .05  signficance  level  may  be  found  in  Table  A-4. 


C.95 


8010  (k  =  2,  df  =  9) 


Therefore,  since  both  calculated  C's  are  less  than  .8010  we  have 
no  evidence  to  reject  the  hypothesis  that  the  variances  are 
homogeneous  for  lots  A  and  C. 


Determination  of  Equality  of  Means  and  Variances  for 
Methods  1  and  Method  2 

This  section  compares  and  contrasts  the  means  and  variances 
obtained  by  each  method  for  a  given  lot.  For  each  lot  the 
following  operations  are  performed; 

a.  An  "F"  test  to  determine  if  the  variances  of  Method  1  and 
Method  2  are  significantly  different. 

b.  A  " t"  test  to  determine  if  the  means  generated  by  Method 
1  and  Method  2  are  signficantly  different. 


Lot  A 


Mean 

Standard  Deviation 

Method 

1 

5.715 

.0496 

Method 

2 

4.871 

.0578 

The  F  ratio  is 

formed 

by  computing 

the  ratio  of  the  sample 

va  r iances ; 


(2-1)  F  =  (.0496) 2/(. 0578) 2  =  .736 


Critical  values  of  F  are  determined  from  Appendix  A. 


The  degrees  of  freedom  for  each  mean  is  9. 
therefore;  f^975  (9,  9)  =  4.03 


and;  ^.025  “  .248 


Since  the  calculated  value  of  F  (i.e.,  .736)  is  in  between  the  two 
critical  F  values,  we  have  no  reason  to  suspect  that  the  variances 
are  different  at  the  5  percent  level  of  significance. 


As  a  result  of  this  calculation,  the  variances  for  Method  1  and 
Method  2  may  be  pooled. 


(2-2)  s2  =  ((m  -1)  si  +  (n2  -1)  ^  )/(ni  +  n2  -2) 

Since  n^  =  n2  =  10,  this  reduces  to: 

S2  =  (  s2  +  si  )/2 

s2  =  {.04962  +  .05782)/2 

s2  =  .005801/2 

Sp2  =  .002901 

Sp  =  .0539 

Using  the  pooled  standard  deviation  fSp  =  .0539)  obtained  by 
pooling  the  standard  deviations  of  Methods  1  and  2  for  Lot  A,  a 
"t"  score  may  be  calculated  to  ,o':ormine  if  the  means  generated  by 
Method  1  and  2  are  signif i*'-jntly  different. 

(2-3)  t  =  (Xi  -  X2)/Sp  (d/ni)  +  (l/n2))-5 

t  =  (5.715  -  5. 871)/. 0539  x  (1/10  +  1/10) -5 
t  =  -  6.47 

This  statistic  should  have  a  "t"  distribution  with  (n^  +  n2 
-2)  or  eighteen  degrees  of  freedom. 

t.975  (18)  =  2.101 

t.025  (18)  =  -  2.101 
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Since  " t"  calculated  is  less  than  -2.101  we  can  say  the  means 
generated  by  method  1  and  2  are  different  at  the  5  percent  level 
of  significance. 


Similar  calculations  should  be  performed  on  lots  B  and  C.  For  the 
case  when  the  variances  for  Methods  1  and  2  are  found  to  be 
nonhomogeneous  the  "t"  score  is  calculated  from: 

(2-4)  t  =  (X^  -  X2)/(s|/Nj^  + 


with  the  degrees  of  freedom  calculated  from 


(2-5) 


df 


(S^/N^  +  sl/n2)^ 

(sf/Nj)2  +  (s|/N2)^ 


+  1 


N2  +  1 


The  results  of  these  calculations  for  Lots  B  and  C  are  stated 
below: 

Lot  B.  Variances  and  means  are  significantly  different  for  each 
method . 

Lot  C.  Variances  are  homogeneous  for  methods  1  and  2  but  means 
are  significantly  different. 


Summary  of  Results  Obtained  in  Part  2 

1.  Cochran's  test  for  homogeneity  of  variances  has  indicated  that 
the  variances  are  homogeneous  for  each  method  only  if  the  data  for 
Lot  B  is  not  included.  The  use  of  Dixon's  range  test  for 
identifying  an  "outlier"  has  been  illustrated  using  the  data  for 
Lot  B,  Method  1.  The  results  of  the  test  indicated  that  an 
outlier  did  exist  in  the  data.  These  two  results  seem  to  indicate 
that  Lot  B  may  be  highly  variable  and  perhaps  heavier  tailed  than 
a  normal  distribution. 


2.  The  means  and  variances  obtained  for  each  lot  using  Methods  1 
and  2  have  been  contrasted  and  compared  for  the  three  lots  in 
question.  The  variances  appear  to  be  homogeneous  for  two  of  the 
lots  and  nonhomogeneous  for  one  lot.  In  all  lots  the  means 
generated  by  the  two  methods  have  been  shown  to  be  significantly 
different  using  the  "t"  distribution. 

3.  The  results  obtained  in  Parts  1  and  2  have  indicated  that  the 
proposed  method  is  less  accurate  than  the  specified  method.  The 
variances  of  each  method  appear  to  be  homogeneous  except  for  the 
results  obtained  with  Lot  B.  However,  if  the  variances  in  Tables 
2  and  3  are  analyzed  using  an  F  test  (as  on  Page  11)  there  is 
evidence  to  indicate  the  variances  may  not  be  homogeneous.  In 
order  to  determine  if  the  variances  are  homogeneous  for  both 
methods  additional  data  is  required.  The  fact  that  Method  2 
appears  to  exhibit  a  constant  bias  may  be  useful,  since  in  some 
cases  the  bias  may  be  of  such  a  nature  that  it  is  predictable  and 
can  be  used  in  characterizing  the  method. 

At  this  point  a  decision  must  be  made  in  regard  to  running  the 
round  robin.  The  data  which  has  been  evaluated  suggests  that  the 
proposed  method  will  not  be  an  equivalent  method.  Based  on  these 
results,  it  is  probably  not  worthwhile  to  conduct  the  round  robin, 
however,  in  order  to  continue  the  example  it  is  decided  to  run  the 
round  robin.  The  results  are  shown  in  Parts  3  and  4. 
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PART  3 


EVALUATION  OF  THE  REPEATABILITY  AND  REPRODUCIBILITY 

OF  EACH  METHOD 

In  this  section  the  bulk  of  the  data  is  treated  in  order  to  get 
estimates  of  the  method  repeatabilities  and  reproducibilities. 

The  data  for  three  lots  is  analyzed  by  five  laboratories  for  each 
method.  This  data  along  with  the  averages  and  standard  deviations 
of  each  lab  is  presented  in  Appendix  C. 

In  order  to  treat  this  data  the  following  assumptions  or  tests 
must  be  made; 

a.  The  data  is  normally  distributed. 

b.  The  data  for  each  laboratory  has  been  inspected  for 
outliers  and  none  are  present. 

c.  The  data  for  a  given  method  and  a  given  lot  (data  from 
five  labs)  have  been  examined  for  homogeneity  of  variance,  and  the 
results  have  shown  that  the  variances  are  indeed  homogeneous. 

The  basic  approach  used  in  this  section  is  to  use  the  technique  of 
analysis  of  variance  (ANOVA)  to  determine  the  variance  between  the 
means  and  the  variance  within  each  lab.  An  "F"  test  is  used  to 
compare  the  two  variances.  If  the  "P"  value  is  found  to  be 
significant,  the  repeatability  and  reproducibility  are  calculated 
by  two  different  equations.  If  there  is  no  significant  difference 
found  by  application  of  an  F  test,  the  repeatability  is  equal  to 
the  reproducibility. 

The  calculations  involved  in  this  procedure  are  illustrated  in  a 
stepwise  fashion  for  the  benefit  of  the  uninformed  reader.  The 
results  of  analysis  of  variance  performed  on  the  six  sets  of  data 
are  shown  in  Appendix  D  The  method  repeatabilities  and/or 
reproducibilities  are  presented  in  Table  6. 


Definitions  Having  to  do  with  Repeatability 
and  Reproducibility 

Laboratory  -  A  laboratory  must  consist  of  a  single  analyst. 

Test  result  -  An  average  of  n  replicate  measurements 

Repeatability  -  A  quantity  that  will  be  exceeded  only  about  5 
percent  of  the  time  by  the  difference,  taken  in  absolute  value,  of 
two  randomly  selected  test  results  obtained  in  the  same  laboratory 
on  a  given  material. 
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Keproaucibiiity  -  A  quantity  that  will  be  exceeded  only  about  5 
percent  ot  the  time  by  the  difference,  taken  in  absolute  value,  of 
two  single  test  results  made  on  the  same  material  in  two 
different,  randomly  selected  laboratories. 


Calculations  Required  for  Analysis  of  Variance  Using  a  One 

Way  Cla ssif ica ion 


The  form  ot  the  data  is  presented  in  Table  5. 


In  order  to  perform  a  one  way  analysis  of  variance,  the  following 
terms  must  be  evaluated: 

t^+/n 


The  pooled  variance  is  calculated  as: 


-  ):(T'.^/ni))/(.'ni  -  k) 


The  mean  square  oetween  categories  is  calculated  as: 


(TV.,/,!.)  -  T4:.^/N 


k  -  1 


where 


=  Number  ot  categories 

=  Number  ot  observations  in  each  category 
=  Sum  ot  tne  observations  in  a  category 
=  Total  number  ot  observations 

=  Within  groups  mean  square  (or  pooled  variances) 


=  Between  groups  mean  square 


Pcxjled  standard  deviation 


Sp 

df 

^ij 

j 

T++ 

X 

Xi 

ss 

MS 


=  Degrees  of  freedom  for  each  sum  of  squares 

=  Observation  number  i  from  column  j 

=  The  sum  of  the  squares  of  all  the  data  points 
in  a  given  group  of  data. 

=  Sum  of  all  the  observations 

=  Grand  mean  =  T++/N 

=  Mean  of  each  category  =  Tj+/n 

=  Sum  of  squares 

=  Mean  square 


The  results  of  these  calculations  are  then  presented  in  a  table 
such  as  Table  6. 
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TABI£  6  -  One  Way  ANOVA 


Sum  of  Squares 

df 

Mean  Square 

Estimate  of 

Between  E(T?+/n^)  - 

k-1 

Sm 

a2  +  no2 

Within  “^^ij 

IT?/"! 

N-k 

4 

Total  rix?j 

/N 

++ 

In  order  to  illustrate 
sample  calculations  are 
data  is  presented  below 
the  next  fiacjc. 

the  calculation  of  the  quantities 
performed  using  the  data  for  lot 
and  the  calculations  usinq  this 

presented  in  Table 
C  method  ?.  The 
data  are  shown  on 

DATA  rOA 

LOT  C  AETMOD  2 

LAB  A 

LAB  B 

LAB  C  LAB  D 

LAB  E 

6.0330 

6.1690 

6.0340 

5.9139 

6.2943 

6.14S3 

6.0700 

6. 1420 

6.0240 

6.0420 

6.0630 

6.017-? 

£ .0530 

5.9-130 

6.0320 

6.1260 

6.Z-,Z9 

5.9423 

5.92;o 

6.2140 

6.1S40 

6.0220 

6.cr.40 

6.2720 

6.1970 

6.1320 

6.2770 

6.2C23 

6.0470 

6.2323 

6.08'<e 

6.1610 

6. 1353 

6.1920 

6.0743 

6.8790 

6.9950 

5.9910 

6.e3£e 

6.2200 

6.1630 

6.1930 

5.8450 

6.0293 

5.9970 

6.1660 

6.3170 

6.0550 

5.9350 

6.0360 

61.015 

61.534 

60.500 

60.134 

61.41 

bach  individual  observation  by  the  five  labs  is  squared  and  these 
values  are  summed: 

(j-ij  =  e.oay^  +  b.i4b^  +  6.063^  5.997^  +  e.osb^ 

=  1856.17116 


Tiie  tive  iao  totals  are  squared,  added  togetner  and  tnen 
divided  by  tne  number  or  replicates  per  Iao. 

(3-,iJ  'rr./n.  =  (61. U5^  +  61.3342  +. .  . +61 . 414^)  /  xO 

=  1853.72907 


Tiie  grand  total  is  squared  ana  divided  by  tne  total  number  ot 
replicates . 

(3-3)  t;’  +  /n  =  (61.015  +  61.534  +...+  bl.4l4)2/50 

=  1855.586648 


i'ne  degrees  ot  ireeuom  used  tor  the  between  groups  mean  square  is 


(3-4)  ur^  =  k  -X 
=  5-1 
=  4 


Tne  degrees  ot  treeoom  used  tor  tne  witnin  groups  mean  square 


(3-5)  ot2  =  N  -K 
=  50  -5 
=  45 


Using  tne  results  ot  (3-1) ,  (3-2) ,  and  (3-3)  the  Sum  or  Squares 

between  groups  ana  witliin  groups  may  be  evaluated. 


?o 


(3-6)  Sum  of  Squares  between  groups  =  -  t^^/n 


=  1855.72907  -  1855.586648 

=  .14243 


(3-7)  Sum  of  Squares  within  groups  = 


=  1856.17116  -  1855.72907 
=  .44209 


Using  the  results  obtained  in  equations  (3-4)  through  (3-7) ,  the 
two  mean  squares  are  now  evaluated: 

(3-8)  Mean  Square  between  groups  =  -  t+^./n)/()<  -  1) 


(3-9)  Mean  Square  within  groups 


=  .14243 
4 

=  3.5606  X  10“2 
_  -  ^T?ynj^)/(N  -  k) 

=  .44209 
45 

=  9.8242  X  10-3 


The  total  Sum  of  Squares  and  degrees  of  freedom,  df,  may  be 
calculated  as  the  sum  of  the  within  groups  and  between  groups 
contributions. 
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(J-j.0)  Total  Sum  ot  Squares  =  .14243  +  .4420y 


(3-ii)  Total  aegrees  ot  treedom  =4+45  =49 


Tue  F  value  is  calculated  by  dividing  the  mean  square  between 
groups  by  the  mean  square  within  groups. 


(3-12) 

F  =  Sm2/sp2 

F  =  (3.5t>06  X  10-2^/ (y,a242  x  10“^) 
F  =  3-624 


Tne  calculated  "F"  value  is  compared  with  the  "F"'  distribution  in 
Appendix  A-3.  "F"  is  presented  as  a  tunction  ot  dtj^  and  dt^ 

(tne  degrees  ot  ireeuom  oi  Sm^  and  Sp2j  ano  ot  tne  level  ot 
signiticance  chosen.  For  this  paper  the  .05  signiticance  level 
IS  cnosen.  Tneretore  F,y5  (dtj^,  dt2)  becomes  F.y^  (4,+5i 
wnere  oi  =  4  (the  aegrees  ot  treedom  associated  with  3m^)  ana 
oL^  =  45  (tne  degrees  ot  ireeaom  associated  with  Sp2) . 


F.y5  (4,  45)  =  2.59 


Since  the  calculated  "F"  value  (3.624)  is  greater  than  F.y5  (4, 

45)  we  may  conclude  that  the  means  ot  the  5  labs  are  unequal  at 
tne  .05  signiticance  level  and  that  the  reproducibility  is 
diiterent  than  tne  repeatability. 

A  computer  program  nas  been  developed  to  do  all  calculations 
described  in  this  section.  Sample  programs  are  shown  in  Appendix 

F. 


Tne  results  oi  these  calculations  are  then  presented  in  Table  7 
Which  nas  the  torm  ot  Table  6.  The  results  ot  each  analysis  ot 
Variance  are  presented  in  Appendix  L).  The  grand  mean  ot  each  data 
set  IS  presented  with  each  analysis  ot  variance. 
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TABLE 


THE  GRAND  MEAN  FOR  LOT  C  NETHOD  2  IS  6.092 


Evaluation  of  Repeatability  and  Reproducibility 
Using  Components  of  Variance 


In  order  to  evaulate  the  method  reproducibility  and  repeatability 
it  is  necessary  to  break  down  Sm^  and  Sp^  into  their 
components  of  variance.  This  is  done  by  utilizing  the  following 
egua  tions . 


(3-13)  Sp' 


(3-14)  Sm2  =  o2  +  no;; 

where  ^2  is  an  estimate  of  the  population  variance  and 

is  an  estimate  of  the  variance  of  the  mean  of  each  lab  about  the 

grand  mean. 

Solving  for  a 2  and  o-^  for  this  example: 


a2  =  9.8242  x  lO'^ 
o2  +  nofe  =  3.5606  x  10-2 

3.5606  X  10~2  _  9.8242  x  lO'^ 


o  2  =  2.5782  X  10-3 
m 

The  repeatability  and  reproducibility  may  now  be  calculated  by  use 
of  equations  (3-15)  and  (3-16): 

(3-15)  Repeatability  =  2.77  x  (o  2/ri).5 

(3-16)  Reproducibility  =  2.77  x  (n  2  +  o^/n)»5 

Solving  these  equations  we  find: 

Repeatability  =  2.77  x  (9.8242  x  10-3/10) *3 
=  .0868 

and  Reproducibility  =  2.77  x  (2.5782  x  10“3  +  9.8242  x  10-3/i0)*5 

=  .1653 
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The  remaining  repeacabxiities  and  reproducibilities  are  calculated 
in  Appendix  E.  The  results  ot  tnese  calculations  are  presentee  in 
Table  b. 


TABLE  b 


Repeatabilities 

Using 


and  Reproducibilities  as  Calculated 
a  One  Way  Classit ication 


Method  1 


Repeatability 

Reproducibility 

Lot  A 

.0397 

.  0397 

aOt  B 

.0236 

.0236 

Lot  C 

.0454 

.0454 

Method  2 

Repeatability 

Reproducibility 

Lot  A 

.0b69 

.  io45 

Lot  B 

.  0b04 

.1547 

iiOt  C 

.Obbb 

.1653 
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PART  4 


A  SINGLE  ESTIMATE  OF  THE  REPRODUCIBILITY  AND  REPEATABILITY  FOR 
EACH  METHOD  USING  A  TWO  WAY  CLASSIFICATION, 

ANALYSIS  OF  VARIANCE  TECHNIQUE 


In  this  section  a  method  of  combining  the  data  for  all  three  lots 
for  a  given  method  is  illustrated.  This  can  only  be  done  if,  for 
a  given  method,  the  variances  for  all  the  lots  are  homogeneous. 

It  is  assumed  in  this  section  that  all  variances  are  homogeneous. 

Once  the  variances  for  all  the  lots  (15  test  results  in  all;  that 
is  three  lots  with  each  lot  analyzed  by  five  different 
laboratories)  have  been  shown  to  be  homogeneous,  a  single  estimate 
of  repeatability  and  reproducibility  based  on  all  the  data  points 
for  that  particular  method  is  possible. 

This  is  done  with  a  two  way  classification  analysis  of  variance 
techn ique . 

The  Analysis  of  Variance  Table  for  a  Two  Way  classification  is 
shown  in  Table  9.  The  nomenclature  used  in  Table  9  is  described 
below: 

n  =  Number  of  replicates 

'  =  Number  of  rows 

c  =  Number  of  columns 

Xj  =  Mean  of  each  column 

Xj  =  Mean  of  each  row 

X^j  =  Individual  mean  of  column  i  row  j 
X  =  Grand  mean  of  all  data  points 

^ije  “  Each  individual  data  point 
Sc  =  Sum  of  Squares  for  Columns 

Sf  =  Sum  of  Squares  for  Rows 

Sj  =  Sum  of  Squares  for  Interaction 

S„  =  Sum  of  Squares  for  Within 

S-p  =  Sum  of  Squares  for  Total 

Sg  =  Sum  of  Squares  for  Subtotal 


TABLE  1;  TWO  WAY  ANALYSIS  OF  VARIANCE 


In  order  to  pertorm  the  ca iculations  involved  in  computing  a  two 
way  analysis  ot  variance,  it  is  necessary  to  arrange  the  data  in 
two  class  1 1 icd ti ons .  The  data  tor  method  one  is  shown  in  Table 
10.  Here,  the  vertical  classitica tions  are  the  labs,  and  the 
horizontal  class i t ica t i ons  ate  the  lots.  Each  set  of  data  tor  one 
lab  and  one  lot  is  reterred  to  as  a  cell.  Each  cell  contains  ten 
repl  1  ca  tes  . 

In  addition  to  presenting  the  data  in  this  tashion,  it  is  also 
convenient  to  torra  a  second  table  which  shows  the  totals  and 
subtotals  ot  eacn  row  and  column.  The  subtotals  are  actually  the 
cell  totals  {i.e.,  the  sum  ot  ten  data  points  tor  a  given  lot  and 
a  given  lab) .  From  these  subtotals,  the  totals  ot  each  row  and 
column  are  then  termed.  The  totals  and  subtotals  formed  from  the 
data  in  Taole  iU  are  shown  in  Table  li. 

Using  a  two  way  analysis  ot  variance  enables  the  statistician  to 
separate  the  total  variance  into  tiie  components  due  to  ditterences 
oetween  la os  and  ditterences  between  lots.  Since  we  do  not  want 
to  include  the  variance  due  to  ditterences  between  lots  in  our 
over  ,  11  estimates  ot  repeatability  and  reproducibility,  this 
tecluiique  is  useful  to  us.  It  enables  us  to  combine  the  data  from 
ail  three  lots  and  thus  get  estimates  ot  the  relevant  variances 
with  higner  degrees  ot  freedom  than  with  the  single  class i t ica t ion 
ANQVA.  It  also  gives  us  a  single  estimate  ot  repeatability  and 
reproducibility  for  each  test  method. 

T'he  toliowing  example  illustrates  the  ca  leu  la  ti  otis  which  must  be 
per  termed  on  these  arrays  in  order  to  evaluate  the  two-way 
analysis  ot  variance.  Once  this  has  been  determined,  use  is  made 
ot  the  components  ot  Variance  approach  to  evaluate  the  appropriate 
componetits  ot  variance  in  order  to  determine  the  method 
r  epea  ta  tJi  1 1  ty  and  reproducibility. 


Calculations  Required  tor  Analysis  ot  Variance  Using 
a  Two  Way  Classification 


Using  the  data  in  Tables  10  and  11  the  toliowing  calculations  are 
per  t  or  m ed  ; 


(4-1)  Sum  ot  Squares  tor  Total: 

S-r  =  +  d./lU^  +...+  fa.U,!9^  +  -  883. 7S7'^/1!50 

S'l'  =  jiUb.3487  -  dJOb. 84.^9 


S'l'  =  1.7Ud8 


Tuo  U«V  CLOSSI^ICATISK-O^TA  or  "ETHOD 


i^onok  aAi/)  cor  i/)9 
r-oof^r“f**r-u>r'r'«o 


tA  l/t  l/k  lA  (A  (A  4/1 U)  iik  i/t 


4P  <» A>  « A  A  9  A 

4A  CA  iO  «■•  (A  t*  •-•  Ui  0)  ^ 
•-•oui*>iA/*io»v  r^or^ 
Ok  CA  00  OD  €A  CA  04  Ol  os  W 


tA  lAtfl  UIII4  lAIA  lAIA  iA 


«»* 

^4ff  «  •*  l»n  C0iDCA9 
v«4Af\j/*>CA«  lAr-  njCA 
^04  U40  fisa>  (A  A  0»tA 


U>IA  U4(A  IA40  lA4A  (OlA 


a  4»«>e»0»<D«<O4V<99 

CAf>  <ijr'«u<or^tAtAoa 
«l  4403  <->00  ^  lA  « 

«  r-r- r-r- fls^  r ‘or- r- 


^  (B/OOOA' 

•-■io<^r-o><xi«u4>«  tu 

AA0»00AAAAAA 


<»  4»  A  «&  A  <0  «ft  A  9 
A)^  «-A<S>  ^aU/CA-V 
U>»  An-<&f  lAOStA-^ 
AA  AA  9  AA 


AA  AIA  A(A  tfktA  AiA 


A  AMDAAAAAAA 


AA AAAAAA AA 


r>OTniAAr>  ACir  A 
A/r-  A^ 

*f>‘r'r“r*r»r*r-Ar^ 


AA  AA  AA  AA  AA 


9999A A9999 

«MA^  ACU  At*  AfUn 
AAn-^m^AAV  ▼ 
AAAAAAAAAA 


A AAAA AAAA A 


99  0»A<W9A  AA 

A  A  Af\i  r- A  A  ▼ 
C9^  -*A  ••A  ^  A  AA 
A  A  -4A  AA  AA  AA 


AAAA AAAA A A 


A^^AAAAAAAA 

AAA^A<VIAt-  A«'4 
AA  AA  Ar-  A  -  r/AA 
rr-AAT^Ar'Arr 


AAAA  A AAA  A A 


A AAAA AAAA A 

Ar- Ar-n»Ar-  4f»AA 
"9  AA  AA  Al%l«*'*ur< 
A AA AA AAAAA 


A AAAA AAAA A 


AAAACPAAAAA 

rur»^»»)  Ai\>  fn  .  • 
AU>  Aa>i’i^rjA  AA 
AAAA AAAA AA 


AA AAAA AAAA 
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(4-2)  Sum  of  Squares  for  Subtotal: 


Sg  =  57.45202/10  +  59.25102/10  +...+  59-08802/10  +  59.90002/^0  -  883.7572/150 
Sg  =  5208.26638  -  5206.8429 
Sg  =  1.42348 


(4-3)  Sum  of  Squares  for  Rows: 

=  288.01102/50  +  296.24802/50  +  299.49802/50  -  883.7572/150 
Sr  =  5208.24531  -  5206.8429 
Sr  =  1.4024 

M-4)  Sum  of  Squares  for  Columns: 

Sc  =  176.69902/30  +  176.87402/30  +...+  176.72202/30  -  883.7572/150 
Sc  =  5206.84388  -  5206.8429 
Sc  =  9.8666  X  lO-'l 

(4-5)  Sum  of  Squares  for  Within: 

This  value  is  found  by  subtracting  the  Sum  of  Squares  for  the 
subtotal  from  the  Sum  of  Squares  for  the  total. 


SS  for  within  =  1.7058  -  1.42348  =  .2823 
(4-6)  Sum  of  Squares  for  Interactions 

This  value  is  found  by  subtracting  the  sum  of  Squares  for  Columns 
plus  the  Sum  of  Squares  for  rows  from  the  Sum  of  Squares  for  the 
subtotal . 

SS  for  Interactions  =  1.42348  -  1.4024  -  9.866  x  10~^  =  .0201 


Table  12:  TUO  UAY  ANOUA-HETHOD 
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Ttie  repeatability  is  tnen  caiculatea  as 
(4-i4j  Kepeatabiiity  =  2.77  x  (^‘/n) 


Tne  reproauc'ibiiity  is  caicuiated  as 

{4-ibj  Reproducibility  =  2,77  x  (o’/n  +  +  o^)  • 

Note  Liie  lot  to  lot  variance,  ,  is  not  included  in  the 

variance  estimate  used  tor  reproducibility . 


Components  or  Variance  Estimates  Using  Two-Way 
ANOVA  tor  MetKod~r 

Equations  (4-iij,  (4-12)  and  (4-10)  may  be  solved  simultaneously 

to  turnish  estimates  ot  the  components  ot  variance  used  in 
ueueriiiining  the  repeatability  and  reproducibility  trom  equations 
(4-14)  ana  (4-lb).  The  procedure  tollows. 

Tne  repeatability  variance  is  estimated  using  tne  mean  square  tor 
witnin  trom  equation  (4-13): 

■  repeatability  =  <’  =  .00209  (4-ib) 

Tne  interaction  variance  is  estimated  using  tne  mean  square  tor 
interactions  uetineu  in  equation  (4-12)  ana  using  the  value  ot 
wnicu  was  uetermined  in  equation  (4-ib)  : 


'  I  ■  .on;''0 


(-5-17) 


wh(-ro  n  -  in,  ■■  -  .(H'?nn 

-  .On?nn)/in 

'vj  =  4.70  >t  lo"'’  (14-lR) 
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Tne  variance  due  to  uitrerences  between  labs  is  estimated 
tne  mean  square  tor  columns  detined  in  equation  (4-iO)  and 
tne  values  ot  and  as  defineo  by  equation  (4-16) 


(4-18) : 


using 

using 

ana 


+  na^  +  rno^  =  .00025 
where  r  =  n  =  10,  +  na^  +  .00251 

=  (.0025  -  .00251)730 

=  -  7.533  X  10“^  (4_iq) 


Tne  sum  ot  tne  interaction  component  ot  variance  ana  the  lab  to 
lab  variance  is  tnen  calculated 

+  Oj  =  4.2  X  10"5  +  (-7.533  x  lo"'’) 

CT^  +  I’t  =  -  3.33  X  10~5  (4-20) 


Since  1’^.  +  0  ,  tne  reproaucibiii ty  is  equal  to  tne 

repeatability.  The  rationale  tor  tnis  is  the  toliowing; 

it  eitner  tne  between-laborator y  or  interaction  variances  is  not 
signiticant,  the  existence  of  random  error  may  cause  either  ot  tne 
respective  mean  square  ratios  to  be  less  tnan  one.  it  this  occurs 
tne  values  or  may  result  in  negative  quantities.  Method 
reproducibility  may  still  be  calculatea,  unless  tne  sum 
ot  +  (i2  is  negative,  in  which  case  the  whole  ot  the  variation 

may  be  assumed  to  oe  aue  to  replication  error  ( )  and  tne 
reproauci bility  is  equal  to  the  repeatability. 
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In  view  ot  tnia  tact  tor  Hetnou  i, 


ceproQucibiitty  =  repeatauiii ty  =  ct2  =  , 

ana  the  repeatability  is  caicuiatea  trom  equation  (4-i4) 

Repeatability  =  2;  77  x 

=  2.77  X  (.00209/10) 

=  .0400 


Reproauci bility  =  Repeatability  =  .0400 
Tne  same  proceaure  is  now  repeatea  tor  Metnoa  2. 


Components  ot  Variance  Calculations  tor  Metnod  2  Us: 

trom  fable  13 


Estimate  ot 


Estimate  ot  oj  ; 


=  .00936 


+  niij  =  .  03bl 


o?  =  .03533  -  .00936 

- TD - 


=  .002597 


Estimate  ot  : 

+  no^  +  rno^  =  .03141 

o2  =  .03141  -  .03533 

- 

ag  =  -1.306  X  iO-4 


00209 


ng  Data 


38 


Now  tiie  repeacdOiiity  anu  reproducioiiity  may  be  caicuiated  trom 
equations  (4-i4j  ana  (4-i5)  . 

iiepeatabiii  ty  =  2.77  x  (  o^/n;*^ 

=  X.77  X  {.OOyib/n)*^  n  =  lO 

=  2.77  X  (.U0y36/i0) 

=  .0847 

KeproauciDiiity  =  /.77  x  (  o^/n  + 

=  2.77  x  (.UU93b/iO  +  (-i.30b  x  iO”'*)  +  (.002397)) 
=  .ib2 

Tue  results  or  tnese  calculations  ate  summarizea  in  Table  14. 


rABi£  14 

Kepeataoiii ties  and  Keproducibiii ti es 
as  Calculated  Usinq  a  Two  way  Ciassii ication 


Kepeatability 
.  0400 


M  et  nod  1 


Reproducibility 
.  0400 


Repeatability 


Method  2 


Heprouuci bil ity 


.  0847 


.  ib2 


PART  5 


CONCLUSIONS 


1.  Method  1  is  more  accurate  than  Method  2.  This  has  been 
demonstrated  in  Part  1. 

2.  The  repeatability  of  Method  2  is  greater  than  the 
Repeatability  of  Method  1  (Parts  3  and  4). 

3.  The  lab  to  lab  variation  is  significant  for  labs  using  Method 
2  but  not  significant  for  labs  using  Method  1  (Parts  3  and  4) . 

4.  In  view  of  2  and  3  no  attempt  has  been  made  to  characterize 
the  bias  exhibited  by  Method  2, 

5.  Method  2  is  not  equivalent  to  Method  1. 
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ARRADCOM  -  Product  Assurance  Directorate 

Subject:  Comparison  of  Test  Methods  for  Determination  of  RDX  in 

Composition  B. 

TO:  Participants  in  Round  Robin  No.  1 

1.  Inclosed  herewith  are  the  following  materials  for  subject 
Round  Robin. 

a.  Three  samples  of  Comp  B  (A,  B,  C) 

b.  Description  of  test  methods 

c.  Instructions  for  collecting  and  reporting  data 

d.  Two  data  sheets 

2.  The  completed  data  sheets  sould  be  returned  by  1  Jan  82. 

3.  The  code  number  for  your  laboratory  is  _ _ _ — 

Very  truly  yours, 

ERIC  R.  BIXON 
ARRADCOM 

Product  Assurance  Directorate 
Building  62 
Dover,  NJ  07801 
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It  is  important  that  all  participants  in  the  Round  Robin  receive 
the  test  procedures  prior  to  receiving  the  samples  and  that  all 
samples  be  received  by  the  participants  within  a  five  (5)  day 
oeriod.  It  is  extremely  important  that  storage  conditions  for 
samoles  (prior  to  testing)  be  identical. 

Individual  testing  by  each  laboratory  should  be  ccxnpleted  within  a 
seven  (7)  day  period  of  receiving  samples.  Note  ambient 
conditions  in  storage  areas  and  laboratory  areas. 


ROUND  ROBIN  NO.  1 

Sample  Data  Sheet  for  a  Given  Method 
q  of  RDX  in  a  lOg  Sample 

LOT  A  LOT  B  LOT  C  REPLICATE  NO. 

-  -  -  1 

2 

3 

-  -  -  4 

5 

6 
7 

-  -  -  8 

-  -  -  9 

-  -  -  10 
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Average  Test  Time,  Mi nutes/Sample 

Working  Time  for  10  Analyses  _ _ 

Total  Elapsed  Time  for  10  Ana  lyses  _ 

Laboratory  Code  No. _  Method  Used_ 

Submitted  by _ _ 

Phone  No.  _ 


Carmen ts  on  Method 


APPENDIX  C 

Data  for  Repeatability  and  Reproducibility  Calculations 
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APPENDIX  D 

Results  of  Analysis  of  Variance  used  in  Calculating 
Repeatability  and/or  Reproducibility 


THE  GRAND  NEAN  FOR  LOT  A  METHOD  1  IS  5.760 


ANALYSIS  or  UARIANCE  FOR  LOT  A  NETHOD 


THE  GRAND  NEAN  FOR  LOT  B  METHOD  1  IS  5.935 


ANALYSIS  OF  UARIANCE  FOR  LOT  B  NETHOD  2 


THE  GRAND  NEAN  FOR  LOT  C  METHOD  1  IS  5.990 


I 


APPENDIX  E 

Calculation  of  Repeatabilities  and  Reproducibilities 

Shown  in  Table  6 


The  following  calculations  use  data  taken  from  the  Tables  in 
Appendix  D: 

Lot  A,  Method  1 

F  =  .735  is  less  than  F.95  (4,  45)  =  2.59  therefore  the 
repeatability  and  reproducibility  are  the  same. 

=  .0020515 

Repeatability  =  2.77  x  ({2.0515  x  10~3)/io).5 
Repeatability  =  .0397 

Lot  B,  Method  1 

F  =  .944,  Repeatability  =  Reproducibility 

s2  =  .0015357 

Repeatability  *  2.77  x  ((1.5357  x  10”3)/io).5 
Repeatability  =  .0236 

Lot  C,  Method  1 

F  =  .859,  Repeatability  =  Reproducibility 

Sp  =  .0026853 

Repeatability  =  2.77  x  (. 0026852/10) • 5 
Repeatability  =  .0454 

Lot  A,  Method  2 

F  =  3.584,  Reproducibility  /  Repeatability 

=  .0098467 


f  8 


.0352863 


Repeatability  =  2.77  x  (. 0098467/10) • 5 
Repeatability  =  .0869 

„2  =  .0352863  -  .0098467  =  2.5440  X  10"3 

-  '  ... 

10 

Reproducibility  =  2.77  x  (c'm  ‘J'/lo)"'’ 

Reproducibility  =  2.77  x  (2.5440  x  10~^  +  (9.8467  x  10"^)/10)*5 

=  .1645 


Lot  B,  Method  2 

F  =  3.701,  Reproducibility  /  Repeatability 

=  .0084229 
=  .0804 


Repeatability 

Repeatability 


2.77  X  (.0084229/10) *5 
.0804 

.031176  -  .0084229 
10 


aj;,  =  2.275  X  10 


Reproducibility 


2.77  X  +  0^/10)-^ 

2.77  X  (2.275  x  10"3  +  . 0084229/10) • 5 


f>e 


.1547 


APPENDIX  F 

Computer  Programs  used  in  Evaluating  ANOVA 
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The  hollowing  flow  charts  are  a  schematic  representation  of  the 
operations  (chemical  analysis  and  data  collection)  and  statistical 
procedures  used  in  this  text. 


The  flow  charts  A,  B,  and  C  correspond  to  Parts  1  and  2  of  the 
Statistical  Guide.  The  proponent  of  the  new  test  method  should  do 
all  the  tests  and  procedures  shown  in  these  flow  charts.  The 
results  of  these  tests  provide  information  as  to  the  accuracy  and 
precision  of  the  new  test  method  as  compared  to  the  old  test 
method.  These  results  form  the  basis  for  maki'^g  a  decision  as  to 
whether  or  not  a  full  scale  round  robin  should  be  conducted. 

Flow  charts  D  and  E  (corresponding  to  Parts  3  and  4)  show  the 
operations  involved  in  evaluating  the  round  robin  data  in  the 
event  tha  the  decision  is  made  to  conduct  the  round  robin.  The 
operations  involved  in  Parts  3  and  4  are  basically  techniques  to 
evaluate  the  method  repeatability  and  reproducibility.  At  first 
glance,  the  reader  may  not  be  abli?  to  distinguish  the  d.fference 
between  these  two  parts.  The  difference  has  to  do  with  the 
degrees  of  freedom  used  in  the  repeatability  and  reproducibility 
estimates.  In  part  3  there  are  three  estimates  of  repeatability 
and/or  reproducibility  for  each  method  basi.‘ci  on  45  degrees  of 
freedom.  Each  estimate  comes  from  ttie  data  for  a  given  lot.  In 
part  4  a  single  estimate  of  the  repeatability  and/or 
reproducibility  with  135  degrees  of  freedom  is  obtained  for  each 
method.  This  single  estimate  is  ba;;<.‘d  on  the  combined  data  for 
all  three  lots. 
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Conclusions;  Both  methods  trust  have  the  same  repeatabilities  and  reproducibilities  to  be  equivalent 
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Chattanooga,  TN  34701 
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Commander 

Newport  Army  Ammunition  Plant 
ATTN:  SARNE-CO 

Newport,  IN  47966 

Director 

Army  Materials  and  Mechanics 
Research  Center 
ATTN:  DRXMR-MQ 

Watertown,  MA  02172 

Commander 

Iowa  Army  Ammunition  Plant 
Silas  Mason,  Mason  &  Hangar,  Inc. 
ATTN:  Technical  Library 

Middletown,  lA  52638 

Commander 

Kansas  Army  Ammunition  Plant 
Parsons,  KS  67357 

Administra  tor 

Defense  Technical  Information  Center 
ATTN:  Accessions  Division 

Cameron  Station 
Alexandria,  VA  22314 

Commander 

US  Army  Research  Office 
BOX  CM,  Duke  Station 
Durham,  NC  27706 

Commander 

Naval  Surface  Weapons  Center 
White  Oak  Laboratory 
ATTN:  Technical  Library 

White  Oak,  Silver  Spring,  MD  29010 

Commander 

US  Naval  Weapons  Center 

ATTN:  Technical  Library  (Code  753) 

China  Lake,  CA  93555 

Comma nder 

Naval  Ordnance  Station 
Technical  Library 
Indian  Head,  MD  20640 
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Director 

US  Army  TRADCO  Systems 
Analysis  Activity 
ATTN;  ATAA-SL 

White  Sands  Missile  Range,  NM  88002 
Commander 

Air  Force  Armament  Development 
and  Test  Center 
ATTN:  AFB  Technical  Library 
Eglin  Air  Force  Base,  FL  32542 
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